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ABSTRACT 

This study assessed the usefulness of the dynamic 
;esting approach to optimize testing procedures by reducing or 
eliminating bias, conceived of as error in measurement attributable 
to factors entering into performance which were not the target of the 
assessment. The study examined: (1) whether the dynamic assessment 
approach yields information which is a more appropriate indicator of 
cognitive competence than standard approaches based on static test 
theory; (2) why and for whom the dynamic assessment approach is 
effective; and (3) for what reason the procedures should be used. The 
sample of 147 fourth- and fifth-grade students from 15 elementary 
schools in the San Bernardino, California, area included 67 Anglo 
Americans, 37 Black Americans, and 43 Hispanic Americans randomly 
assigned to standard, verbalization, or elaborated feedback test 
conditions* Measures of cognitive and perceptual functioning, 
cognitive style and personality (e.g., impulsivi ty-ref lect ivi ty , 
motivation and planning, and school achievement) included Raven's 
Coloured Progressive Matrices, the Matching Familiar Figures Test, 
the Cattell Culture Fair Test of "g," Harter's Perceived Competence 
Scale for Children, and Trail Making and Visual Search Tasks. It was 
found that the testing condition which seemed to be most effective in 
increasing performance of all groups was verbalization; a type of 
compensatory effect for initially observed differences was brought 
about by subject verbalization. (CM) 
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Chapter I 
The Problem 

1 .0 Introduction 

A significant concern for our era relates to the con- 
ceptualization of what constitues appropriate assessment of cognitive 
and perceptual processes. The debate has focused on key terms such as 
"test bias" or "test fairness." Several different approaches to this 
issue have been taken. These range from the classical test fairness 
concept of equal prediction elaborated by Cleary (1968), Cleary, 
Humphreys, Kendrick, and Wesman (1975), and Eichorn and Bass (1971), 
to criteria involving group membership and cultural variables 
(Darlington, 1971), and to the incorporation of sociological data in 
arriving at performance level (Mercer & Lewis, 1975). 

Recently, Flaugher (1978) has cautioned that test bias can result 
from many factors and influences. Focus on j use one aspect may be my- 
opic and ignore the breadth of the issues involved. He did suggest, 
however, that the development of "tailored testing" would perhaps pro- 
vide a solution to one very important factor: bias related to the 
atmosphere in which the test is administered. The notion of tailored 
testing can be extended to the use of a learning oriented approach to 
assessment (Resnick, 1979). Through such an approach (referred to 
here as dynamic testing) analyses can be made of how putative 
motivational, personality, and cognitive style factors interact with 
assessment approaches to yield performance data. This allows for an- 
alysis of bias between and within ethnic and/or cultural" groups and 
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clarification of individual and group differences in terms of psy- 
chological and information processing variables. In addition, dynamic 
assessment provides a basis for modifications in the classroom to 
incorporate learning characteristics of the child in the teaching 
strategy and/or teaching methodology used. 
1 J Bases of Dynamic Testing 

An assumption often made in assessment cf cognitive functioning 
is that test scores obtained in the traditional , standardized manner 
serve as adequate estimates of the cognitive competence of the person 
or group of persons tested. The validity of the assumption is based 
on the notion that performance is a veridical measure of competence. 
This suggests that intraindi vidual variations in information proces- 
sing, resulting primarily from intellectual and/or personality 
factors, play relatively unimportant roles in test outcome. The re- 
lationship between subject- and task-independent variables is ignored 
(cf. Detterman, 1979). 

The static test approach and the assumptions behind it has been 
challenged by several authors (Brown & French, 1979; Feuerstein, 1979; 
Guthke, 1977; Schmidt, 1971). It is argued that testing procedures 
must be developed and applied which adequately take intraindi vidual 
variability into account and reduce the potential discrepancy between 
measured cognitive functioning and cognitive competence (Bortner & 
Birch, 1970). This calls for a dynamic testing approach, where 
modifications in testing procedures can be analyzed as they affect 
performance levels and interact with sources of individual 
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variation without diminishing predictive validity. In this approach, 
variations in testing procedure can be built directly into the test 
situation itself or implemented through training outside of the actual 
testing. The latter requires subsequent assessment of the effects of 
the training on a transfer test and is thereby more difficult for the 
practitioner to implement than the former approach. We favor the 
former, incorporating the modifications in the procedures into the 
testing situation, for pragmatic and methodological reasons. 
Pragmatically, it is convenient and directly applicable for use by 
psychological practitioners. Methodologically, it avoids problems re- 
lated to the measurement of change (Cronbach & Furby, 1970). 
1 # 2 Testing Conditions Leading to Increased Levels of Performance 

A series of studies were carried out to isolate and assess the 
effect of several types of commonly used variables which can be 
incorporated into testing procedures and be independently manipulated 
(Carlson & Wiedl, 19 76; Carlson & Wiedl, 19 79; Wiedl & Carlson, 19 77). 

The testing procedures isolated involve different techniques of 
optimizing cognitive functioning. Thus, controlled comparison between 
types of approaches often used, but heretofore not systematically an- 
alyzed in the dynamic testing approach, was made. The dependent 
measure was the Rav;-r Coloured Progressive Matrices (CPM), a test of 
nonverbal intelligence (Raven, 1976). The CPM was chosen as it is 
considered to be a relatively "culture fair" test and a good measure 
of general intelligence. The subjects tested were 434 second- and 
fourth-grade children from the general population. The testing con- 
ditions (C) employed were: 

ERIC ° 
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C^. Standard Instruction, according to the procedures outlined 
by Raven. 

C 2 . Verbalization During and After Solution, which requires 
the child to describe the main stimulus pattern prior to searching for 
the correct ansver and then, after a particular alternative is chosen, 
to explain why he had the choice he did. 

C 3 . Verbalization After Solution, which involves the child de- 
scribing the reasons for his choice afcer the choice is mace. 

C 4 . Simple Feedback, where the child is informed only if he is 
correct or incorrect after his choice. 

C 5 . Elaborated Feedback, which involves, in addition to simple 
feedback, an elaboration by the experimenter of the reasons why the 
chosen answer was correct or incorrect. The principles involved in 
the task were pointed out. 

C 6 . Elaborated Feedback plus Verbalization During and After 
Solution, which is a combination of conditions two and five. That is, 
it involves the child's verbalization of the pattern to be completed, 
followed by solution and the child's explanation for the reasons for 
solution, and elaborated feedback by the experimenter informing the 
child of the correctness of his response and explaining the principles 
invol ved in the task. 

In all cases the child's initial response, regardless of con- 
dition, was scored. 

For the second-grade children, Conditions 2, 5, and 6 led to 
higher levels of performance than were found for Conditions 1, 3, and 
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4. The most effective conditions were 5 and 6. For the fourth-grade 
children, Conditions 2 and 6 led to improved performance. A trend in 
this direction was noted under Condition 5. Thus, three conditions 
for younger children and two procedures for older children were found 
which appeared useful for improving test scores, i.e., bringing per- 
formance closer to actual competence. 

Beyond ascertaining the main effects of the testing conditions on 
total score CPM performance, closer analysis of the results was ob- 
tained by analyzing the interactive effects of performance with dif- 
ferentiations which the CPM allows in administration and scoring. The 
test can be administered in either the normal booklet form or in a 
puzzle, board-form version. In the former, covert processes must be 
used by the subject as he searches for solution. In the latter, overt 
trial -and-error procedures can be used. In scoring the CPM, three 
distinct subgroupings of items can be used. These were distinguished 
through factor analysis in a previous study (Wiedl & Carlson, 1 976). 
the factors isolated were: simple pattern completion, pattern com- 
pletion through closure, and reasoning by analogy. (See Carroll & 
Maxwell, 1979 for a review and Carlson & Jensen, 1980 for further 
analysis.) 

Results of analysis of these differentiations showed that the 
puzzle version led to higher levels of performance for the second- 
grade but not the fourth-grade children. The salient item groups were 
simple pattern completion and pattern completion through closure. The 
effects of the optimizing testing conditions could be traced mainly to 
those items requiring abstract thinking. 

er|c iu 



On the basis of these results, two types of studies were con- 
ducted. In one type, the goal was to replicate and extend the above 
findings by investigating whether or not the salient testing con- 
ditions have the effect of optimizing performance for children from 
special populations. In the other type, in addition to extending 
general applicability, the goal was to assess the compensatory effect 
which the procedures could have in reducing differences between par- 
ticular groups of children. 

1 .3 Children with Learning Difficulties 

In studies involving learning handicapped, low IQ children (Carl- 
son & Wiedl, 1979; Wiedl & Carlson, 1980), basically the same testing 
paradigm as described for the first study was used. Space limitation 
prevents full description of the investigations. By way of summary, 
however, it was demonstrated that for learning handicapped subjects 
testing procedures C^ and Cg are appropriate and the effects could 
be demonstrated for items requiring abstract reasoning. This result 
is of special significance as the requirements of tasks of logical 
thinking are particularly difficult for low IQ children. 

The most important results described in the preceding section are 
summarized in Table 1 on the following page. 

1 .4 Personality and Cognitive Style Variables 

The above presentation of the dynamic assessment approach has 
been concerned primarily with the validation of the dynamic assessment 
procedure in terms of effects on mean performance scores. Beyond 
this, however, it is important to investigate how and to what extent 



Table 1 



Summary of Salient Testing Conditions vith Different Groups of Subjects 



Testing Conditions 



Groups 

of Subjects 



Differentiated C- 
Verbalization 4 



Elaborated C. 
Feedback 



Verbalisation C- + C c 
+ Feedback 4 * 



Board* Form 



Item Requirements 



T p a r 



T p o r 



T p c r 



T s> c r 



Regular School 
Children 2 nd grade 
4th grade 



O 
9 



O 



» • • 



Children with Learning 

Difficulties 

(avg. 4th grade) 



4 0" slight 
• - marked 



• • • 



+■§■ f • to'.al test score 

p » pattern completion items 



o - closure items 
r - reasoning items 
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learning characteristics (personality and cognitive style) of the 
child interact with dynamic testing approaches. 

Several studies have stressed the interaction of learning and 
personality characteristics. Three investigations will be mentioned 
here that relate to one particular personality (style) characteristic: 
impul sivity-reflectivity. Bush and Sweck (1975) have concluded that 
task demands influence strategies children use who display reflective 
information processing while impulsive children seem to be unaware of 
the strategy demands of the tasks. 

In a study relating the dynamic assessment approach to impulsive 
children, Wiedl (1979) hypothesized that the performance of impul si ves 
could be made equal to that of reflectives by modified, more 
appropriate testing procedures. The subjects were 150 second-grade 
chidlren, average age 7. 6 years. The Matching Familiar Figures test 
(MFF) was administered to all children to assess degree of 
impul si vity- reflectivity. Based on their MFF performance the 
subjects were divided into two groups, impulsives or relectives. The 
classification was obtained by median-split on the error score of the 
test. The CPM was administered under testing Conditions 1 , 2, 5, and 
6. 

The results showed that impulsives scored lowest on the CPM under 
the standard testing condition (Cj). No differences between the 
impulsives and reflectives were detected under Conditions 2 and 5. 
That is, significant gains in performance were noted for the 
impulsives, but not for the reflectives when conditions of verbaliza- 
tion (C 2 ) and elaborated feedback (C 5 ) were employed. Under 
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Condition 6, no compensatory effect was detected for the impulsive 
children. In addition to the MFF error score classification, children 
were also categorized as impulsives or relectives by a median-split on 
latency to response. For this classification, similar compensatory 
effects attributable to testing condition were detected in matrices 
performance. 

In a study designed to ascertain how differences in test per- 
formance were reduced under the verbalization (Cg) and elaborated 
feedback conditions (C 5 ), Wiedl and Bethge (1980) analyzed eye 
movements for impulsive and reflective children defined by MFF latency 
scores. Several types of eye-movement measures were made. But re- 
gardless of eye-movement measures taken, the same basic result was 
found: Under conditions of dynamic assessment, and Cg, the eye 
movement patterns of the impulsive and reflective children were about 
the same. Such was not the case with the Raven Matrices administered 
under Cj. The following exemplified the general findings. 
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1 # 5 Compensation with Regard to Ethnic/Racial Differences: 

Anglo-Americans, Black-American, Hispanic-American Children , 
The potential compensatory effects of the dynamic testing ap- 
proach in assessing cognitive capabilities of three different ethnic 
groups, Anglo, Black, and Mexican-American, were studied (Dillon & 
Carlson, 1978). Several investigations have been carried out com- 
paring performance of thes2 groups on tests like the Raven matrices 
and various Piaget tests. Most results indicate that Black children 
tend to perform approximately one standard deviation below Anglos, 
with Mexican-American performance half-way between. It was hypo- 
thesized that the testing conditions shown to be salient in the pre- 
vious studies would compensate for factors which may contribute to the 
relatively low Black and Mexican-American performance. 

The tests used were Matrices and Order of Appearance taken from a 
recently developed Piaget battery (Winkelmann, 19 75). The testing 
procedures involved three conditons: Standard (Cj), Verbalization 
During and After Solution (C 2 ), and Verbalization During and After 
Solution plus Elaborated Feedback (Cg). 

The tasks were administered to 189 children, representing three 
American ethnic groups: Anglo, Black, and Mexican-American. The sub- 
jects ranged from five to ten years of age. Randomization and as- 
signment to testing conditions was carried out for three age levels: 
5-6, 7-8, and 9-10 years. This resulted in a 3x3x3 randomized block 
design, with three levels of test administration, three age levels, 
and three ethnic group classifications. There were seven subjects in 
each eel 1 . 
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Table 2 



Means and Standard Deviations of Matrices for Three Age Groups, 
Ethnic Groups and Testing Conditions (C). 



_ag'e: 5-6 



age: 7-8 



age: 9-10 



Anglo 
•Mean 

SD 



1.29«--->.71< — =>-2.43 
.95 1.11 2.23 



3.43<-->2.43<— >4.29 
2.37 1.51 .95 



4 . 14< — >6 . 00«- ->6 . 29 
2.80 1.73 1.60 



•Hex- Am 

Mean 
SD 



.57* — 1. 71<f—> 2. 43 
.98 1.70 2.23 



2.14^ ->2. 71$ >4.00 

.90 1.50 1.41 



4.29<f 55.71^- ->6.14 

2.29 1.38 1.07 



Black 



.57« ->.86< — >2.00 
.79 .85 1.83 



1.43<^->2.71<^->4.14 
1.40 . 1.60 .90 



2.71«£— >5.72<- ~>6.00 
1.38 1.25 1.00 



— - difference not significant 
difference significant 
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Since results for the Matrices and Order of Appearance tests 
turned out to be essentially the same, only those for the Matrices 
will be given. Inspection of the means and standard deviations re- 
veals a marked decline in differences in performance between the 
groups. The differences ranged from approximately two-thirds of a 
standard deviation between Anglo and Black performance under Condition 
1, to essentially zero under Condition 6. Although this could be at- 
tributed to a ceiling effect for the older group (means of just over 6 
from an 8-1 tern test), such an interpretation cannot be made for the 
younge- and middle groups (means around 2 and 4 respectively). The 
results are summarized in Table 2 on the following page. 
1 .6 Criterion-Related Validity 

The issue of test fairness is closely related to the criterion- 
validity of the test. The most commonly used application is pre- 
dictive validity. If a test, i.e., testing procedure which yields the 
scores, is "fair," the relationship between the predictor and criter- 
ion contains minimum random error and the random error is constant be- 
tween specific groups tested. 

In a previous study (Carlson & Wiedl, 1979), differing patterns 
of corrlations were detected between CPM performance obtained through 
modified testing procedures and measures of school achievement. 
Similarly, Carlson and Dillon (1979) found that prediction of school 
success was enhanced through the application of dynamic testing pro- 
cedures. Unfortunatley, specific comparisons of predictive validity 
of several measures of Level II intelligence and school success have 
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not been made employing dynamic assessment approaches with children of 
different ethnic/racial background. 
1 . 7 Summary 

The approach of applying optimizing testing procedures to the 
normal population as well as to special groups of children was de- 
monstrated. Restrictions and differentiations in the effects of tes- 
ting procedures, test format, and task demands were found. The ap- 
plicability of certain testing procedures for purposes of assessing 
intellectual performance of normal and special school (learning handi- 
capped, low IQ) children was shown. This indicates that specific forms 
of performance optimization can lead to more accurate, thus fairer, 
assessment of intellectual functioning. It was demonstrated that 
specific dysfunctional individual characteristics such as impulsivity 
can be compensated for under certain testing conditions. This 
underscores the necessity of critically analyzing how variations in 
testing procedure can interact with certain individual differences. 
The implication here is that the practitioner must be aware of these 
interactions and under which circumstances performance may be 
increased for some children, while decreased for others. 

The indication that compensatory effects can be obtained in as- 
sessing between-ethnic group differences is of practical and the- 
oretical significance. Extension of the approach by employing other 
measures of Level II intelligence (Jensen, 1973) to larger samples 
with control over factors such as socio-econommic status is necessary 
before firm conclusions can be reached, however. 

ERLC 
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Chapter II 
The Approach 

2.0 Specific Objectives and Methods 

The basic objective of the project was to assess the usefulness 
of the dynamic testing approach as a way of reducing or eliminating 
test bias. Bias here has two definitional bases: (1) where a 
significantly large proportion of variance in test performance can be 
attributed to other than "cognitive" factors; and (2) where dif- 
ferential prediction, i.e., over- or underprediction , of academic per- 
formance for any particular group occurs. Bias is not defined in 
terms of mean differences in performance between groups. Rather, it 
is conceived of as error in measurement attributable to factors en- 
tering into performance which are not the target of the assessment, 
e.g., when noncognitive and/or style variables unduly affect cognitive 
performance. 

2.1 Research Questions 

The research was designed to shed light on three interrelated 
questions addressing issues related to assessment of cognitive 
functioning and learning characteristics of children. 

1. Does the dynamic assessment approach yield informa- 
tion which is a more appropriate indicator of cognitive 
competence than that obtained by traditional, standard 
approaches based on static test theory? 
Corollary A: Do specific testing conditions 
differentially affect the performance of children 
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of different social and/or ethnic 
background? 

If, as expected, the answer to the above questions are positive, 
the following question becomes: 

2. Why and for whom is the dynamic assessment approach 
effective? Stated differently: What are the sources 
of intramdi vidual variation, and how may they be ac- 
counted for by personality and cognitive style variables? 

The third question addressed the issue of predictive validity and 
can be stated in the following way: 

3. For what reason should dynamic assessment procedures be 
em pi oyed? 

The objective was to assess the impact of dynamic assessment on 
the very real problem of predicting school and curricular success. 
2.2 Methods 

The organization of this section will be along the following 
lines. First, the sample will be described. Second, the general pro- 
cedures followed will be outlined. Third, the instruments used will 
be given and procedures followed in their administration specified. 

2.2.1 The Sample 

Originally the plan of the study was to attain a sample of 225 
fourth-grade children: 75 Anglo-American, 75 Black-American, and 75 
Hispanic-American. The children were to be matched on socio-economic 
level. Our goals in sample size and selection were not fully realized 
due to logistic and "political" reasons. 
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The final sample consisted of 147 fourth- and fifth-grade 
children from 15 elementary schools serving the San Bernardino, 
California area. The schools were selected from low-, middle- and 
high-SES areas in order to insure that the widest possible SES range 
would be included in the sample. Thei e were 67 Anglo-American 
children (29 males, 38 females), mean age 10 years, 4 months, standard 
deviation 9 months; 37 Black-American children (20 males, 17 females), 
mean age 10 years, 8 months, standard deviation 11 months; and 43 
Hispanic-American children, (18 males, 25 females), mean age 10 years, 
5 months, standard deviation 10 months. 

One the factors involved in not obtaining the total sample size 
we had originally planned on was due to the unwillingness of minority 
parents to allow their sons or daughters to participate in this pro- 
ject. Written permission, in the form of a signed informed-consent 
letter, was required before a child was included in the study. The 
San Bernardino Unified School District left it to the discretion of 
the principals at the individual schools whether or not a given school 
would participate in the project. For various reasons, the principals 
of schools that had large minority populations were less willing to 
cooperate than principals in schools serving a more heterogeneous 
group of children. Nonetheless, we were able to obtain sufficient 
numbers of minority students to carry out most of the analyses 
pi anned. 

From each ethnic group, subjects were randomly assigned to one of 
three testing conditions: Cj standard; C 2 verbalization; C 3 



0 

ERIC 



2x 



17 

elaborated feedback. The numbers of subjects in each condition are 
shown in the following table 

Test Condition 



Ethnic Group 


CI 


C2 


C3 


Total 


Angl c 


23 


22 


22 


67 


Black 


11 


12 


14 


37 


Hispanic 


14 


17 


12 


43 


Total 


48 


51 


48 


147 



2.2.2 Procedures 

Due to the large amount of time required for testing and the 
lin.ited testing time available per testing session at most schools, a 
minimum of three sessions was necessary for complete testing of one 
subject. Total testing time per subject was approximately 2 1/2 to 3 
hours. The majority of this time was used to administer the two tests 
of cognitive and perceptual functioning: Raven's Matrices and the 
Cattell Culture Fair Test "g" (CFT). Each of these measures took 
approximately one hour to administer. The Raven Matrices were always 
administered before the CFT, and usually during the first session. 
Typical testing sessions were as follows: 

Session 1 Session 2 Session 3 

Raven Matrices CFT Harter Perceived 

Matching Familiar Trail -Making Competence Scale 

Figures Test (MFFT) Visual Search 
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All testing was done individually by trained graduate students. 
Specific administration protocols for each measure will now be pre- 
sented. 

2.2.3 Measures of Cognitive and Perceptual Functioning 
1 . Raven's Matrices . 

Raven's Coloured Progressive Matrices (CPM) consists of 
three sections (A,Ab,B) of 12 items each. Each item consists 
of a main stimulus pattern with a piece missing from the 
lower right hand corner. The subject's task is to select 
from the alternatives presented underneath the main stimulus 
pattern the "piece" that correctly completes the pattern. 
Factor analytic studies of the CPM have revealed item 
groupings that require simple pattern completion, pattern 
completion through closure, and concrete and abstract re- 
asoning by analogy (Weidl & Carlson, 1976; Carlson & Jensen, 
1980). 

Reliability and validity estimates of the CPM are gener- 
ally reported to be high. Carlson and Jensen (1981) 
found the CPM to have equally high reliability for each of 
three ethnic groups: Anglo, Black and Hispanic. 

In this study, CPM items were supplemented by section C 
(12 items) of the Standard Progressive Matrices (SPM). The 
format of the SPM items is essentially the same as the CPM 
items; although the items are not in color and 8 alternatives 
are presented rather than 6. Section C of the SPM was 

ERIC 
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administered after section A, Ab, and B of the CPM had been 
completed. Thus the test consisted of a total of 48 items. 
All 147 subjects completed the Raven's CPM and section C of 
the SPM. The test conditions and specific administration 
protocols associated with them are presented below: 

C-| - Standard Instruction - The standard protocol set 
forth in the CPM manual (Raven, 1965) was used. The 
tester turned to the first problem in the test booklet 
(item Al), pointing out to the subject the main stimulus 
pattern, noting that it had a piece missing from the 
lower right hand corner. The child's attention was then 
directed to the six answer alternatives below. He/she 
was told that only one of the six alternatives was cor- 
rect and completed the pattern above. He/she was 
asked to say which one he/she thought was the correct 
one. If the child chose the correct piece, the tester 
replied "Yes, that's the right one," and explained why 
that was the right one and why the other five 
alternatives were incorrect. If the child picked a dis- 
tractor, the tester said "No, that's not the right 
one," and pointed out to the child the correct piece. 
This was followed by an explanation of why it was the 
correct one, and why the one the child had picked was 
incorrect. 

The child was then told to write down the number of 
the correct piece on his/her answer sheet, and that on 
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each page of the booklet there was a similar problem. 
He/she was asked to work through the booklet individual- 
ly, and to write down the number of the piece 
he/she thought was correct in the appropriate place on 
his/her answer sheet. When the child completed the CPM, 
the tester showed him/her Section C of the SPM and told 
him/her that these were like the puzzles he/she just 
did, except that they were not in color, and he/she now 
had eight pieces to choose from. No further instruction 
was given. 

C 2 - Verbalization During and After Solutior . The 
tester showed the child item Al and asked him/her to de- 
scribe the main stimulus pattern, paying attention to 
the details of the pattern. E aided S in "sharpening" 
his/her description. The child was then asked to choose 
from the answer alternatives below the pattern that 
he/she thought was correct and would complete the pat- 
tern. The child was then asked to explain why he/she 
thought the chosen alternative was correct. Again E 
aided S "sharpen" his/her response. For this sample 
item, the tester informed the child if his/her choice 
was correct or incorrect and why. The child was then 
instructed to work through the rest of the booklet, but 
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very carefully describe to the tester the main stimulus 
pattern before giving an answer. He/she was asked to 
always describe why he/she thought the alternative 
he/she chose was the correct one. After the first sam- 
ple problem, the tester gave the child no feedback as 
to the correctness of his choice. The tester recorded 
the child's responses on an answer sheet. When the CPM 
was finished, Section C of the SPM was given, 

C 3 - Elaborated Feedback , As in conditions Cj, the 
tester showed the child item AT and pointed out that the 
main stimulus pattern had a piece missing from the lower 
right hand corner, and that his/her task was to select 
from the six alternatives below the one which would cor- 
rectly complete the pattern. If the child chose the 
correct piece, the tester said "Yes, that's the right 
one," and explained why it was correct. If the child 
chose a distractor, the tester replied "No, that's not 
the right one," and explained why it was an incorrect 
response. E then pointed out the correct piece, and 
explained why it was correct. This same procedure was 
followed for the remaining items, except that the 
experimenter did not point out the correct piece if the 
child's response was wrong. The tester always made an 
effort to explain to the child the principles involved 
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in the solution of the problem. Often the child chose 
the correct response after the experimenter had ex- 
plained the principles involved. However, in all three 
conditions, only the child's initial response was 
recorded by the experimenter, i.e., the response given 
before any feedback. 
Cattell's Culture Fair Test of "g" (Scale 2, Form A ) 
Cattell's CET was administered in the second or third testing 
session, and in the same manner as the Raven's Matrices. 
The CET is composed of four subtests: "Series Completion" 
(12 items), "Classification" (14 items), "Matrices" (12 
items), and "Conditions," or topological relations (8 items). 
The Series, Matrices, and Conditions subtests have 3 
practice items each. Classification has two items. 

The Cattell CFT is a well standardized, highly reliable 
intelligence test. Total scores on the CFT correlate mod- 
erately with other intelligence tests, including Raven's 
Matrices. 

The same general procedures as described above for 
administration of the Matrices under standard, verbalization, 
and elaborated feedback conditions were followed. The tester 
worked through the practice items of each test with the 
child, informing him/her whether the child was correct or not 
and why. Due to repeated absences, three subjects did not 
receive the CFT. 
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2.2.4 Measures of Cognitive Style and Personality 
1. Matching Familiar Figures Test (MFF) 

The MFF (Kagan et al . , 1963) was used to assess individual 
differences in the cognitive style dimension "impulsivity- 
reflecti vity." The test consists of 12 items. 
Each item has a picture of a familiar object, e.g., a house. 
Below the picture, six alternatives, one of which is iden- 
tical to the picture above, are given. The subject's 
task is to point to the picture below which he/she thinks is 
the one identical to the picture "as quickly and correctly as 
you can." The tester records latency to the subject's first 
response, and the number of errors made on each item. Test 
scores used for analysis are mean latency and total errors 
across all twelve items. 

Impulsive children are defined as those who have fast 
(below the group median) latencies, and who make many (above 
the group median) errors. Reflective children are defined as 
having slow (above the group median) latencies, and few 
(below the group median) errors. A "double median split" on 
time and errors is usually performed to identify impulsive 
and reflective children. However, since this procedures has 
methodological difficulties, and results in a considerable 
loss of information. For analysis in this study, MFF time 
and error were treated as separate continuous variables. 
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All 147 subjects completed the MFF. 
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2. Harter's Perceived Competence Scale for Children 

The Perceived Competence Scale for Children is a measure of 
motivation recently developed by Harter (1979a; 1979b). The 
model upon which the scale is based is that a child's per- 
ception of his/her competence is an important factor in 
his/her intrinsic motivation to be effective, curious, prefer 
optimal challenge, and engage in independent mastery at- 
tempts. The scale assesses four domains of perceived com- 
petence: (1) "cognitive competence," which emphasizes 
academic performance; (2) "social competence, 11 which stresses 
social and peer group relationships; (3) "physical com- 
petence," which focuses on physical prowess; and (4) 
"general self-worth," which refers to being sure of one's 
self, being happy with the way one is, etc* The scales have 
been shown to have moderate to high reliability, and have 
been used with children 6 to 12 years of age. 

Each domain consists of seven items. E read each item 
aloud to the subject. For example, one item from the cogni- 
tive competence domain is: "Some -kids often forget what 
they learn BUT other kids can remember things easily." The 
child is asked to decide if he/she is more like the first 
child described, or more like the second child described. 
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Once the child makes this decision, he/she is asked if the 
description is "really true," or only "sort of true" for 
him/her. Each item is scored on a scale of 1 to 4; the lower 
score reflecting greater perceived competence. 

Due to repeated absences, one child was not given the 
Harter scales. 

Trail Making and Visual Search Tasks 

The Trail -Making and Visual Search tasks are used to measure 
planning ability. The role of planning and decision making in 
the performance of cognitive tasks has been established by 
Luria (1973), and expanded to include psychometric con- 
siderations by Das (1980). A good planner exhibits the fol- 
lowing characteristics: (1) "looks before he leaps," (2) 
checks for errors, (3) takes all factors into account before 
making judgment, (4) evaluates feedback and acts accordingly, 
and (5) can generally come up with strategies to approach a 
problem. Das (1980) has found that planning ability affects 
cognitive performance, but is factorially independent from 
the impul sivity-reflectivity dimension and simultaneous or 
successive modes of information integration. 
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Das (personal communication) presents the following model 
incorporating planning and decision making in cognitive 
functioning. 



External Factors of Presentation- 
Visual, Auditory, etc. 



Method of Presenting Stimuli* 
Successive-Simultaneous 



Sensory Registration 



— i 

Central Processing 




The Visual Search task consists of 48 geometric shapes, 
letters and numerical sets. Due to limitations in testing 
time, a subset of 14 items was used in this study. Each item 
is presented on a transparency, at the center of which is a 
circle which contains an exact copy of the one "target" 
figure (or letter or number) contained elsewhere on the 
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transparency. The subject's task is to find and place 
his/her finger on the figure which matches the one in the 
circle. The tester records the elapsed time for the subject 
to search the transparency and find the target figure (search 
time), and the time it takes for the subject to move his/her 
finger and touch the appropriate figure (movement time). 
Search time is measured from when the transparency is il- 
luminated to when S moves his/her finger from a button on 
the illumination box. Movement time is the time of the bal- 
listic from the button to the figure on the transparency. 
Two timers are used. Movement times were not included in any 
of the analyses, as this variable has not been shown to be 
central to planning. Search times, on the other hand, are 
the most important measure on this task. Thus these data 
were analyzed. 

The Trail Making task presents the subject with stimulus 
sheets containing numbers and letters distributed randomly. 
The subjects task is to draw lines connecting the letters and 
numbers in the correct sequenc - % e.g., 1-A, 2-B, 3-C, etc. 
There are two parts to the test The tester records with a 
stopwatch the elapsed time for *»'*:e subject to complete each 
part of the test. Total time on both parts is recorded and 
used for analysis. 

Due to repeated absences, the Trail Making task was not 
administered to one subject. Due to repeated absences and 
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equipment failure, complete and usable visual search data 

were collected or only 131 subjects. 
2.25 Measures of School Achievement 

In order to assess the "predictive validity" of the cognitive 
ability measures administered under the different test conditions, two 
measures of school achievement were obtained. These were standard 
scores on reading and mathematics from the California Test of Basic 
Skills (CTBS) administered in April 1982. Reading test scores were 
available for 130 subjects; math test scores for 122 subjects. 
2.3 Summary 

A total of 147 fourth- and fifth-grade children representing 
three ethnic groups participated in this study. Measures of cognitive 
and perceptual functioning, impulsi vity-reflectivity , motivation, and 
planning were administered. School achievement data, in the form of 
CTBS reading and math scores, were obtained. 

Analysis of the data will address (1) the effect of testing con- 
ditions (static vs. dynamic) on total Raven and Cattell scores, as 
well as on subtests and item groupings of these measures; (2) the re- 
lationship of ethnic group classifiction to Raven's and Cattell per- 
formance under the different test conditions; (3) the effects of non- 
target variables (impulsivity-reflectivity, motivation, planning) on 
Raven and Cattell performance under the different conditions; and (4) 
the "predictive validity" of the cognitive ability measures 
administered under standard vs. elaborative testing conditions. 
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Chapter III 
The Results 

3.0 Organization 

The results will be organized and presented in three sections. 
The first section will address the effects of testing condition per se 
on Raven and Cattell performance for the three ethnic groups in the 
study. The second section will address the relationship between non- 
cognitive individual difference variables (impul sivity , motivation, 
planning) and performance on Raven's matrices and Cattell 1 s CFT for 
each testing condition. The final section will present a comparison 
of the predictive "power" of Raven's Matrices and Cattell 's CFT 
administered under the different conditions. In Sections 1 and 2, 
Raven item groupings and the Cattell subtests will be treated as de- 
pendent variables in addition to total scores on these two measures. 

3.1 Relationship to Test Condition and Ethnic Group Classification 
to Raven's Matrices Performance 

Table 1 presents the means, standard deviations, and coefficent 
alpha (Cronbach, 1951) reliability estimates for total Raven Matrices 
scores separately by test condition and ethnic group classification. 
As will be noted, the reliabilities tend to fluctuate slightly across 
testing condition and ethnic group classifictioh. Nonetheless, they 
remain generally high. 
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Table 1 

Raven Matrices Total Score Means, Standard Deviations, and 
Internal Consistency (Coefficient Alpha) Reliability Estimates as 
a Function of Test Condition and Ethnic Group Classification 



Test 



Condition 




Anglo 






Black 




Hispanic 






X 


SD 




X 


SD 


r kk 


X SD 


r kk 


c i 


36.9 


5.4 


.82 


34.0 


5.8 


.82 


31.4 6.4 


.85 




40.8 


3.3 


.61 


36.9 


6.1 


.86 


34.2 5.3 


.78 


r 


37.5 


4.5 


.72 


34.2 


5.5 


.75 


32.3 4.7 


.66 



Cj= Standard Administration; Verbalization; Elaborated Feedback 



Potential effects of the testing conditions on performance wasr 
analyzed by analysis of variance. Table 2 presents the results of the 
3 (test condition) x 3 (ethnic group) analysis of variance performed 
on Raven total scores* 
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Table 2 

Test Condition (A) x Ethnic Group (B) Analysis of 
Variance on Raven Total Scores 



'Source df MS F 

Test Condition (A) 2 135.97 5,20** 

Ethnic Group (B) 2 443.28 16.94** 

A x B 4 2.99 > 1 

Error 138 26.17 



Inspection of Table 2 reveals significant main effects for test 
conditon (A) and for ethnic group (B). The A x B interaction did not 
approach statistical significance. Scheffe Dost hoc comparisons were 
performed to further analyze the significant main effects for test 
condition and ethnic group. These comparisons showed that Anglos 
scored significantly higher than either Hispanics (p <.01) or Blacks 
(p <.01), while no significant difference between Hispanic and Blacks 
was detected. These results are presented graphically in Figure 1. 
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0 H 1 1 L 

CI C2 C3 

TESTING CONDITION 



Fig. 1. Performance of Three Ethnic Groups on Raven's Matrices 
Administered Under Three Testing Conditions. 
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Q<- 1 1 I 

Anglo Black Hispanic 

ETHNIC GROUP 



Fig. 2. Effect of Testing Condition on Raven Matrices Performance 
for Three Ethnic Groups. 
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The multiple comparisons for test condition revealed that per- 
formance under C2 (Verbalization) was superior to performance under 
Cj (standard administration) (p<.05), and marginally superior to 
performance under C3 (elaborated feedback) (p = .058). C] and 
C3 were not significantly different from each other. Figure 2 
graphically presents mean total Raven scores under the three test con- 
ditions for each ethnic g r p. 

The results of the test condition x ethnic group ANOVA can be 
summarized as follows: Regardless of ethnic group classification, 
performance is highest under (verbalization), and lowest under 
Cj (standard administration). Performance under C 3 (elaborated 
feedback) is only slightly higher than performance under Cj« 
Regardless of test condition, Anglos performed significantly higher 
than both Blacks and Hispanics. Blacks tended to have higher total 
Raven scores than Hispanics under all three test conditions. However, 
the difference only approaches statistical significance. 

The items of Raven's CPM were divided into two groups which have 
been shown to require different processes or strategies for solution: 
Pattern Completion and Closure items and Concrete or Abstract Re- 
asoning by Analogy items. The groupings were based on previous factor 
analytic studies of the CPM (see Carlson & Jensen, 1980; Wiedl & Car- 
lson, 1976). Section C of the Standard Progressive Matrices (SPM) was 
analyzed separately as a third subgroup of items. Means and standard 
deviations for the three item subgroups are presented in Table 3 
separately for the test condition x ethnic group categories. 

ERIC 3D 
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Table 3 

Raven Item Subgroup* Means and Standard Deviations 



as a Function 


of Test 


Condition 


and Ethnic 


Group 


Classification 








Ethnic Group 






Test Condition 


Anglo 




Black 




Hispanic 






X 


SD 


J 


SD 


I 


SD 


(Ci) 

n a 4" 4" ri 

ra l tern 
Completion 


16.2 


1 .0 


15.5 


2.1 


14.5 


2.1 


Reasoning 


2.8 


1.7 


2.5 


1 .9 


2.1 


1.7 


Section C 


7.5 


1 .9 


6.2 


2.4 


5.7 


2.5 


(c 2 ) 

Pattern 
Compl etion 


16.7 


0.6 


15.9 


1.4 


15.8 


2.5 


Reasoning 


4.1 


1.1 


3.3 


1.6 


2.8' 


2.0 


Section C 


8.4 


1.5 


7.3 


1.9 


6.2 


2.0 



Separate analyses of variance were carried out for each Raven 
item subgroup. The independent variables were test condition and 
ethnic group. The results are summarized in Table 4. 
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Table 4 

Summary Table for Results of Analyses of 
Variance on Raven Item Subgroupings 



Source Pattern Completion Reasoning Section C 

Test Condition (A) xx x ns 

Ethnic Group (B) xx x xx 

A x B ns ns ns 



X F value significant, p<.05 
xX F value signifianct, p <.01 

Significant main effects for test condition were found for pat- 
tern completion and reasoning item subgroups. Significant main ef- 
fects for ethnic group were found for pattern completion, reasoning 
and Seciton C item groupings. None of the test condition x ethnic 
group interactions was significant. Scheffe post hoc comparisons were 
performed to further analyze the significant main effects for test 
condition and ethnic group classification. These comparisons are sum- 
marized in Table 5. 
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Table 5 

Summary of Scheffe Post Hoc Comparisons for Test Condition 
and Ethnic Group* on Raven Matrices Item Subgroups 

Item Subgroup 

Comparis on Pattern Completion Reasoning Section C 

(A) Test 

Condition C2>C3 ( p< . 01 ) C2>C1 (p<.01) x 



(b) Ethnic 

Group A>H (p<.01) A>H (p<.01) A>B (p<.05) 

A>H (p<.01) 

*A=Anglo; B=Black; H=Hispanic 

The results of the multiple comparisons for test condition on the 
item subgroups are similar to those for test condition performed on 
total Raven scores. The verbalization condition (C2) was found to 
lead to higher pattern completion scores than C3 ( p< . 01 ) and higher 
reasoning item scores than Cj (p<.01). Since the omnibus F test for 
test condition c,< Section C items was nonsignificant, no post hoc com- 
parisons for test condition were performed on this item subgroup. 

The results of the multiple comparisons for ethnic group on the 
Raven subgroup items are also similar to those reported for Raven 
total scores. Specifically, Anglos performed higher on all three item 
subgroups than Hispanics ( p< . 01 ; p<.05; p<.05, respectively); and 
higher on section C items than Blacks (p<.05). 
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Figure 3, on the following page, shows the mean scores on the 
three item sugroups as a function of test condition and ethnic group. 

The graphs reveal additional information concerning trends in the 
data not revealed by the ANOVA or the multiple comparisons. Specific- 
ally, they show that for the pattern completion items, performance 
tends to be poorest under C3 for all three ethnic groups; albeit not 
significantly poorer than performance under C^. C 2 tends to be 
the most efficacious condition for pattern completion items for all 
three ethnic groups and for reasoning and section C items for the An- 
glo and Black subjects. Interestingly, the Hispanics 1 best 
performance on the reasoning and section C items was under C3. 
(Again, this is a trend, approaching statistical significance.) 

When the results of the Raven item subgroup analyses are combined 
with the results of the analysis of total Raven scores, the following 
conclusions can be msde concerning the relationships between ethni- 
city, testing condition, and Raven's matrice performance: (1) Re- 
gardless of item type or test condition, Anglos perform better on 
Raven's matrices than either Blacks or Hispanics; (2) C 2 appears to 
be the most effective condition for maximizing performance, although 
C3 is equally effective or slightly more so than C 2 for Hispanic 
subjects on the more difficult reasoning and Section C items; (3) both 
elaborative conditions (C 2 and C3) lead to enhanced levels of 
performance on the more difficult reasoning and section C items for 
all ethnic groups. However, elaborated feedback (C3) appears to 
"hinder" performance on the relatively easy pattern completion 
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Fig. 3. Performance of Ethnic Groups on Raven Matrices Item 

Subgroupings Administered Under Three Testing Conditions. 
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items, since performance on these items tended to be lower than 
performance under standard administration procedures for all three 
ethnic groups, 

3,1 ,1 Relationship of Test Condition and Ethnic Group Classification 
to Cattell CFT Performance 
Table 6 presents the means, standard deviations, and coefficient 
alpha reliability estimates for total Cattell CFT scores separately by 
test condition and ethnic group classification. 

Table 6 

Cattell CFT Total Score Means, Standard Deviations, and Internal 
Consistency Reliability Estimates as a Function of Test Cndition 
and Ethnic Group Classification 

Anglo Black Hispanic 

Test ~ ~ 

Condition -X- SD r.. X SD r. . X SD 



C ] 27.7 6.1 .80 25.7 5.8 .77 26.8 4.6 .63 

C 2 32.8 4.8 .73 28.6 5.8 .79 28.4 5.9 .78 

Co 30.8 4.9 .74 25.9 5.7 .74 26.2 4.5 .61 



C^ Standard Administration; C 2 = Verbalization; C 3 = Elaborated Feedback 
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Table 7 shows the results of the test condition x ethnic group 
analysis of variance performed on Cattel 1 total scores. 

Table 7 

Test Condition (A) x Ethnic Group (B) Analysis of 
Variance on Cattel 1 Total Scores 



Source 


df 


MS 


F 


Test Condition (A) 


2 


121 .55 


4.23* 


Ethnic Group (B) 


2 


219.79 


7. 65** 


A x B 


4 


21 .27 


< 1 


Error 


135 


28. 72 





*p <.05 
**p<.01 

Inspection of Table 7 reveal s results nearly identical to those 
reported in Table 2 for the ANOVA on Raven total scores. Significant 
main effects are shown for test condition (A) and ethnic group (B). 
The A x B interaction is not statistically significant. Scheffe post 
hoc comparisons for test condition show that performance under C 2 is 
significantly higher than performance under C^ (p <.01). The dif- 
ferences in mean Cattell scores between C^ and C 3> and C 2 and 
C3 are not statistically significant. 
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Scheffe post hoc analyses for ethnic group indicate that Anglo 
subjects scored significantly higher than both Black (p <.01) and 
Hispanic (p <.05) subjects on Cattell total scores. Black and 
Hispanic Cattell scores were not significantly different. Figures 4 
and 5 graphically present Cattell total socres as a function of test 
condition and ethnic group, respectively. 

Each of the four subtests of the Cattell CFT was analyszed 
separately. The means and standard deviations on these subtests are 
presented separately for teach test condition by ethnic group category 
in Table 8. 

Table 9 summarizes the results of the test condition x ethnic 
group ANOVAs performed one each of the Cattell subtests. 

Inspection of Table 9 reveals significant main effects for test 
condition on two of the Cattell subtests: matrices and "conditions," 
or topological relations. A significant main effect for ethnic group 
was found on the series completion and matrices subtests, with the 
difference on the "conditions" subtest approaching statistical 
signifiance (p = .08). The test condition x ethnic group interaction 
on the "conditions" subtest also approached significance (p - .06). 

Scheffe post hoc comparisons were performed or. those subtests for 
which there were significant main effects. These comparisons are sum- 
marized in Table 10. 

The comparisons presented in Table 10 show that on the series 
completion subtest Anglo subjects performed significantly higher than 
both Black ( p<. 01 ) and Hispanic subjects (p<.05). On the matrices 
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Fig. 4. Performance of Ethnic Groups on Cattell CFT Administered 
Under Three Testing Conditions. 
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Anglo Black Hispanic 

ETHNIC GROUP 



Fig. 5. Effect of Testing Condition on Cattell CFT Performance on 
Three Ethnic Groups. 
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Table 8 

Means and Standard Deviations for Cattell CFT Subtests as a 
Function of Test Condition and Ethnic Group Classification 

Ethnic Group 

Subtest 

and 



Test Condition Series Classification Matrices Conditions 





5T 


SD 


X 




SD 


X 


SD 


X 


SD 


Anglo C-] 


8.5 


1 .5 


6 


.7 


2.2 


8.1 


1 .8 


4.3 


1 .9 


Anglo C 2 


9.3 


1 .8 


7 


.7 


2.1 


9.9 


1.2 


6.0 


1 .6 


Anglo C 3 


8.7 


1 .6 


7 


.6 


1 .9 


9.0 


1.8 


5.5 


1.5 


Black C 1 


6.9 


2.5 


6 


.7 


1.5 


7.4 


2.2 


4.7 


2.1 


Black C 2 


7.8 


2.0 


7 


.4 


2.0 


8.3 


1.8 


5.2 


1.5 


Black C 3 


7.2 


2.4 


6 


.9 


2.2 


7.6 


2.2 


4.2 


1 .2 


Hispanic 


8.1 


2.3 


6 


.9 


2.0 


7.3 


1.4 


4.6 


1.9 


Hispanic 


7.6 


2.0 


7 


.2 


1.9 


8.0 


2?.6 


5.6 


2.0 


Hispanic C 3 


7.8 


1.1 


7 


.0 


1 .8 


7.9 


1 .8 


3.5 


1 .6 
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Table 9 

Summary Table ^or Results of Analyses of 
Varian-e on Cattell CFT Subtests 



Source Series Classification Matrices Conditions 

Test Condition (A) ns ns x xx 

Ethnic Group (B) xx ns xx p=.08 

A x B ns ns ns p=.06 

X F value, p < .05 
XX F value, p < .01 



Table 10 

Summary of Scheffe Post Hoc Comparisons for Test Condition 
and Ethnic Group on Cattell CFT Subtests 



Cattell Subtest 



Comparison 


Series 


Matrices 


Conditions 


(A) Test 

Condition 


X 


C2>Cl(p<.01) 


C2>Cl(p<.01) 


(B) Ethnic 
Group 


A>B(p<.01) 


A>B(p<.01) 


C2>C3(p<.05) 




A>MA(p<.05) 


. A>MA(p<.01) 


X 



0 
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subtest, C 2 led to higher performance levels than Cj (p<.01). 

Again, Anglo subjects performed significantly higher than both Black 

(p<.01) and Hispanic (p<.01) subjects. On the subtest "conditions", 

the verbalization test condition, C 2 . led to higher performance 

levels than either C] (p<_.01) or C 3 (p<.05). 

Figures 6 through 13 graphically depict performance on the four 

Cattell subtests as a function of test condition (Figs. 6-9) and 

ethnic group (Figs. 10-13). 

The analyses performed on the Cattell subtests indicate that the 

main effect for test condition found on Cattell total scores is 
largely due to the facilitative effect of verbalization on performance 
on the matrices and topological relations ("conditions") items. The 
significant main effect for ethnicity on Cattell total scores can be 
traced to the superior performance of Anglo subjects on the series 
completion and matrices items. As Figures 7 and 11 show, there was 
very little difference in performance on classification items 
attributable to either test condition or ethnic group category. 

In general, the elaborative testing conditions, verbalization and 
feedback, led to significantly higher levels of performance for Anglo 
subjects. Verbalization led to higher levels of performance for Black 
and Hispanic subjects as well. However, elaborated feedback did lit- 
tle to improve the performance of the minority groups. 
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Fig. 6. Performance of Ethnic Groups on CattelTs CFT Series 

Completion Items Administered Under Three Testing Conditions. 



0 

ERIC 



53 



49 



oo 10 

LU 

o y 
o 

00 8 



LU 



-z. 
o 

< 



00 
00 

< 
_J 

o 



7 
6 
5 
4 
3 
2 




^Hispanic 



Anglo 
Black 



C2 



C3 



TEST CONDITION 



Fig. 7. Performance of Ethnic Groups on Cattell's CFT 

Classification Items Administered Under Three Testing 
Conditions. 
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Fig. 8. Performance of Ethnic Groups on Cattell's CFT 

Matrice Items Administered Under Three Test Conditions. 
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Fig. 9. Performance of Ethnic Groups on CatteH's CFT 

"Conditions" Items Administered Under Three Test 
Conditions. 
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Fig. 10. Effect of Testing Condition on Cattell's CFT 

Series Completion Items for Three Ethnic Groups. 
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Fig. 11. Effect of Testing Condition on Cattell's CFT 
Classification Items for Three Ethnic Groups. 
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Fig. 12. Effect of Testing Condition on CatteTTs CFT 
Matrice Items for Three Ethnic Groups. 
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Fig. 13. Effect of Testing Conditions on Cattell's CFT 
"Condition" Items for Three Ethnic Groups. 
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3.1.2 Summary 

In summary, the results in this section show significant dif- 
ferences in performance on Raven's Matrices and Cattell's CFT. 
These were attributable to test condition as well as to ethnic group 
classification. Analysis of the main effects revealed that, for test 
condition, the verbalization procedure (C 2 ) was the most ef- 
ficaceous, leading to the highest performance levels on both Raven's 
Matrices and Cattell's CFT. Additional analyses indicated that the 
verbalization test condition led to enhanced performance specifically 
on the pattern completion and analogical reasoning items of the 
Raven's, and on the matrices and topological relations items of the 
Cattel 1 . 

Analysis of the main effect for ethnic group classification 
indicated that, regardless of test condition, Anglos performed at 
higher levels on both Raven's matrices and Cattel Ts CFT than either 
Blacks or Mexican/Americans. This difference held for all three item 
subgroups of Raven's matrices; but was true only for the series com- 
pletion and matrices subtests of the Cattel 1 . 
3.2 The Relationship of "Nontarget" Variables to Performance on 

Raven's Matrices and Cattell's CFT Administered under Standard 

and Elaborative Testing Conditions 

Analysis of the effects of impulsivity, motivation, and planning 
on Raven and Cattell performance will be presented as follows. First, 
potential ethnic group differences on all the "nontarget" variables 
will be assessed by one way analyses of variance. Second, the non- 
target variables will be correlated with Raven and Cattell scores 
separately within each testing condition. Where no ethnic group 
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differences were found, for the nontarget variables, all subjects 
within each condition will be combined for the correlational analyses. 
Where significant ethnic group differences are detected, the 
correlational analyses will be performed separately for the ethnic 
groups. 

The following questions will be addressed: (1) Do ethnic groups 
differ on any of the impulsivity, planning, or motivational variables? 

(2) Do any of the impulsivity, motivational, or planning measures 
correlate significantly with Raven matrices (or with Raven item sub- 
groups) or Cattell's CFT (or with Cattel 1 subtests)? (3) If some of 
these measures do correlate with Raven and Cattell performance, does 
the relationship hold for all test conditions? (4) If some of the 
variables are related to Raven matrices and/or Cattell CFT performance 
under at least one test condition, are they variables which are dif- 
ferentially related to ethnic group classification? 
3.2.1 Ethnic Group Classification, Impulsivity, Motivation and 
PI anning 

Table 11 presents the means, standard deviations, and sample 
sizes on all "nontarget" variables separately for each ethnic group. 

One way analyses of variance were performed for each of the nine 
nontarget variables. The results of these ANOVAS are sumarized in 
Table 12. 

Inspection of Table 12 indicates a significant main effect for 
ethnic group on only one variable: Trail Making. Ethnic group 
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Table 11 



Nontarget Variable Means, Standard Deviations, and 
Sample Sizes for Three Ethnic Groups 



1 

Impulsi vity 




Anglo 






Black 




Hispanic 




X 


SD 


N 


X 


SD 


N 


X 


SD 


N 


MFFT 


10.4 


5.8 


67 


9.4 


•5.7 


37 


9.4 


5.6 


43 


MFFE 


9.3 


4.0 


67 


10.6 


4.6 


37 


10.9 


4.3 


43 


2 

Planning 




















TM 


179 


43 


67 


195 


43 


36 


202 


57 


43 


Mean VS 


2.79 


0.77 


61 


2.49 


0.64 


32 


2.90 


0.72 


38 


SD VS 


1 .67 


0.78 


61 


1 .39 


0.65 


32 


1 .77 


0.76 


38 


3 

Motivation 




















CC 


15.0 


3.8 


67 


15.6 


4.5 


36 


15.3 


4.4 


43 


SC 


14.6 


4.9 


67 


15.2 


4.5 


36 


14.7 


4.3 


43 


PC 


16.2 


4.8 


67 


16.8 


4.7 


36 


15.9 


4.3 


43 


GSW 


14.4 


4.0 


67 


15.6 


4.4 


36 


15.7 


4.2 


43 



Impulsivity Variables - MFFT, Matching Familiar Figures Test Time (Seconds) 

MFFE, Matching Familiar Figures Test Errors 



"Planning Variables 



Motivation Variables 



TM, Trail Making Time (seconds) 

Mean VS, Mean Visual Search Time (seconds) 

SD VS, Standard Deviation of Visual Search Time 

CC, Harter's Cognitive Competence Scale 
SC, Harter's Social Competence Scale 
PC, Harter's Physical Competnece Scale 
GSW, Harter's General Self Worth Scale 
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Table 12 



Summary Table for Analyses of Variance on Nontarget Variables 



Measure/Variable 


df 


MS (Ethnic Group) 


MS (Error) 


F 


Impulsivity 










MFFT 


2 


17.3 


32.9 


< 1 


MFFE 


2 


41.1 


18.1 


2.27 


Planning 










TM 


2 


7884 


2264 


3.48* 


Mean VS 


2 


1 .55 


0.53 


2.92 


SD VST 


2 


1.37 


0.56 


2.47 


Motivation 










CC 


2 


4.83 


17.38 


< 1 


SC 


2 


6.24 


21.63 


< 1 


PC 


2 


6.70 


21 .29 


< 1 


GSW 


2 


26.96 


17.18 


1 .57 



* 

p < .05 
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differences approached statistical significance on the other two plan- 
ning variables (mean and standard deviation of visual search time), 
and on MFF errors. 

Scheffe post hoc comparisons were performed on Trail Making 
scores, and revealed that Hispanic subjets took significantly more 
time to complete the trail making task than Anglo subjects (pjs.05). 
Hispanic subjects did not differ significantly from Black subjects; 
nor Black subjects from white subjects. 

3.2.2. Correlation of Nontarget Variables with Raven and Cattell 
Performance 

Within each testing condition, all ethnic groups were combined 
and scores on the nontarget variables (exception: Trail Making) were 
correlated with Raven total scores and item subgroups; and with Cat- 
tell total scores and the Cattell subtests (Tables 13 and 14, re- 
spectively). Because of the significant ethnic group difference on 
the Trail Making variable, separate sets of correlations were 
calculated between Trail Making and the measures of cognitive and 
perceptual functioning for (1) Hispanic subjects; (2) Anglo subjects; 
(3) Anglo and Black subjects; and (4) Hispanic and Black subjects. 
For the sake of clarity of presentation, only statistically 
significant correlations are reported in Tables 13 and 14. 

Inspection of Table 13 reveals that MFF Time is significantly re- 
lated to total Raven's performance under all three test conditions. 
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Table 13 

Correlations between Nontarget Variables and Raven 
Matrices Scores as a Function of Test Condition 

Test Condition 



Cl c 2 c 3 



Variable 


RAVTOT 


PC 


REAS 


SEC C ' 


RAVTOT 


PC 


REAS 


SEC t 


RAVTOT 


PC 


REAS 


SEC C 


MFFT 


.24* 


.35* 


ns 


ns 


.23* 


ns 


ns 


.35** 


.24* 


ns 


.43** 


ns 


MFFE 


-.42** 


-.41** 


-.26* 


-.28* 


-.30* 


ns 


-.24* 


-.38** 


-.38** 


-.31* 


-.39** 


ns 


TM (A) 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


-.39* 


ns 


ns 


TM (H) 


ns 


ns 


ns 


ns 


ns 


ns 


-.41* 


ns 


ns 


ns 


ns 


ns 


TM (A + B) 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


-.38* 


ns 


ns 


TM (H + B) 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


Mean VS 


ns 


ns 


-.35** 


ns 


-.33* 


ns 


-.37** 


-.37** 


ns 


ns 


ns 


ns 


SD VS 


ns 


ns 


ns 


ns 


ns 


ns 


-.45** 


ns 


1 1 o 


1 1 O 


n c 


ns 


cc 


ns 


.25 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


sc 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


PC 


r.s 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


GSW 


ns 


ns 


ns 


-.26* 


-.23* 


ns 


ns 


-.29* 


ns 


ns 


ns 


ns 



**p< . 01 
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However, the tendency to respond quickly appears to be differentially 
related to performance on different item types depending on the method 
of test administration. Under standard testing conditions (C^), 
"impulsive" (quick responding) children seem to do poorly on the pat- 
tern completion items, whereas this variable does not seem to be re- 
lated to pattern completion performance under C 2 °r C3. Under the 
verbalization condition (^2), highest performance was found on Sec- 
tion C items for those subjects who were less "impulsive" (slower re- 
sponders). Similarly under elaborated feedback test conditions 
(C3), performance on the reasoning items were best for the less 
"impulsive" subjects. 

MFF Errors are more systematically related to performance under 
the three testing conditions. Errors on the MFF are moderately 
negatively correlated with Raven total scores for all three con- 
ditions. Errors are also significantly related to all the item sub- 
groupings under Cj; and with all the item subgroupings except pat- 
tern completion (under C 2 ) and Section C items (under C3). 

The correlations of Trail Making time with Raven's matrices were 
computed for Anglo and Hispanic subjects separately, as well as for 
Anglo and Black subjects combined and Hispanic and Black subjects 
combined. Table 13 reveals that regardless of ethnic group, Trail 
Making time is not systematically related to Raven performance under 
any test condition. There are no significant correlations between 
Trail Making and Raven performance under Cj. For Trail Making 
is moderatley correlated (r=-.41) with the reasoning items for 
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the Hispanic subjects only. Under C3, Trail Making is significantly 
(r=-.38) related to performance on the pattern compoletion items for 
the combined Anglo and Black samples. In both of these cases, 
subjects who had fast trail making times tended to have higher scores 
on these item groupings. 

Planning, as assessed by Visual Search, appears to be related to 
Raven performance under the verbalization test condition. Mean VST is 
moderately correlated (r=-.35) with total Raven score under this con- 
dition, as well as with the reasoning (r=-.37) and Section C (r=-.37) 
item subgroups. The standard deviation of VST is moderately (r=-.45) 
related to the reasoning items here as well. 

There appears to be little relationship between the motivation 
measures and Raven performance. Not surprisingly, the social and 
physical competence scales do not correlate with Raven total score or 
with any of the item subgroup; ngs under any test condition. The 
cognitive competence scale has a single significant correlation 
(r=.25), that being with the pattern completion items under C^. 
However, it is likely that this is a spurious correlation since it is 
in the wrong direction, i.e., it indicates that individuals who have 
low perceived cognitive competence tend to have higher scores on the 
pattern completion items. The "General Self Worth" scale has three 
significant correlations with Raven Performance, These correlations 
indicate that those subjects who have a realtively high sense of "self 
worth" tend to do better on Section C items (under test conditions 
Cl (r=-.26) and C 2 (r=-.29) and also higher total Raven scores 
(under C 2 (r=-.23)). 
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In summary, the impulsivity variables (MFFT and MFFE) appear to 
have the strongest and most systematic relationship to Raven per- 
formance. This relationship apparently obtains regardless of test 
condition, although different item groupings tend to be differentially 
related to MFF Time due to test condition. The planning variables, 
especially Visual Search, seem to be related to performance under the 
verbalization test condition, and may relate particularly to the per- 
formance of Hispanic subjects on the reasoning items. The motivation 
variables show little relationship to Raven performance under any 
condition, except for the "General Self Worth" scale which has several 
significant, although relatively low magnitude, correlations with 
Raven performance. 

Inspection of Table 14 reveals that MFF Time has only one 
significant correlation (r=.24) with CattelTs CFT, that being for the 
motives subtest under C 1# MFF Error, on the other hand, is 
significantly related to Cattel 1 total score under all three test con- 
ditions, as well as to the matrices and topological relations subtests 
regardless of test condition. MFF Error appears to be the most sensi- 
tive predictor of either Raven or Cattell performance of the two 
impulsivity variables. 

Trail Making is more highly related to Cattell performance than 
to Raven performance. Individual differences on this planning vari- 
able seem to significantly effect the performance of Anglo subjects 
under Z>2* The elabaorative conditions seem to have a compensatory 
effect for those who have the poor planning ability. 
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Table 14 

Correlations between Nontarget Variables and Cattell's 
CFT Scores 1 as a Function of Test Condition 

Test Condition 



C l c 2 C 3 



Variable 


CATTOT 


SI 


S2 


S3 


S4 


CATTOT 


SI 


S2 


S3 


S4 


CATTOT 


SI 


S2 


S3 


S4 


nrr 1 


ns 


ns 


ns 


.24* 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


MFFE 


-•35** 


ns 


ns 


-.30* 


-.28* 


-.37** 


-.32* 


ns 


-.24* 


-.38** 


-.43** 


ns 


ns 


-.50** 


-.41** 


TM (A) 


-.43* 


ns 


-.40* 


-.67** 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


TM (H) 


ns 


ns 


ns 


ns 


ns 


-.45* 


-.56* 


ns 


ns 


-.64** 


ns 


ns 


-.56* 


ns 


ns 


TM (A ♦ B) 


ns 


ns 


-.30* 


-.36* 


ns 


ns 


ns 


ns 


ns 


ns 


-.42** 


-.29* 


ns 


-.34* 


-.48** 


TM (B + H) 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


-.51** 


ns 


ns 


ns 


ns 


ns 


Mean VST 


-.28* 


ns 


-.27* 


-.26* 


ns 


-.38** 


ns 


-.36** 


-.35** 


ns 


ns 


ns 


ns 


ns 


ns 


SD VST 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


-.28* 


-.27* 


ns 


ns 


ns 


ns 


ns 


-.31* 


CC 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


SC 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


PC 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


GSU 


ns 


-.28* 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 



l CATTUT= Cattell Total Score; Sl= Series Completion Subtest; S2= Classification Subtest; 
S3= Matrices Subtest; S4= "conditions", or Topological Relations Subtest. 

*p<.05 
**p<.OI 
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That is, the significant correlations observed between planning and 
Cattell performance under (r s range from -.43 for Cattell total 
to -.40 and -.67 for the classification and matrices subtests, re- 
spectively) become nonsignificant under C2 and C3. On the other 
hand, Trail Making does not appear to be related to Cattell per- 
formance for Hispanic subjects under Cj, although it appears to be a 
factor under the verbalization condition for these subjects. Trail 
Making seems to be quite strongly related to the performance of 
Hispanic subjects on 'both the series completion and topological 
relations items under Z^* For the combined Anglo and Black sample, 
Trail Making seems to be primarily related to Cattell performance 
under the elaborated feedback condition, with significant correlation 
with Cattell total score (r=-.42) as well as with series completion 
(r=.29), matrices (r=-.34) and topological relations (r=-.48) sub- 
tests. For the Black and Hispanic combined sample, Trail Making is 
significantly related (r=-.51) only to the topological relations items 
under Z^. Thus, the relationship between the planning variable, 
Trail Making, and Cattell performance appears to vary as a function of 
both test condition and ethnic group classification and is much more 
sytematical ly related to Cattell performance than to Raven 
performance. 

Visual Search time is related to total Cattell performance under 
both Cj and C2 (rs=-.28 and -.38, respectively). Both mean Visual 
Search time and the standard deviation of Visual Search time are cor- 
related significantly with^the classification (rs= -.36 and -.28, 
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respectively) and matrices (rs=-.35 and -.27, respectively) subtests 
under C 2 . Mean visual search time is also significantly related to 
Cattell total score and to the classification (r=-.27) and matrices 
(r=-.26) subtests under Cj. 

In sum, the planning variables, Trail Making and Visual Search, 
are mostly related to performance under C2» and appear to be a 
factor especially affecting the performance of Hispanic subjects in 
this condition. 

The motivation variables appear to be even less related to Cat- 
tell performance than they are to Raven performance. There is only a 
single significant correlation between any of the motivation scales 
and Cattell performance, that between the "general self worth" scale 
and series completion items under Cj_ (r=-.28). Since only three 
significant and interpretable correlations between Raven performance 
and the Harter scales were with the "general self worth" scale, it 
would appear that "general self worth" is a fairly global measure of 
intrinsic motivation and is a better predictor of performance on 
cognitive ability measures than the other, more narrowly focused, 
scales. 

3.2.3. Summary 

The following summarizes the results presented, specifically with 
respect to the questions posed at the beginning of the section: 

(1) Significant ethnic group differences were found on only one 
nontarget variable (Trail Making), although there were 
trends toward ethnic group differences on the other 
planning variable (Visual Search) and on MFF Errors. 
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(2) Of the three sets of nontarget variables, the impulsivity 
(MFF Time and MFF Error) and planning (Trail Making and 
visual search) variables show the strongest relationship to 
Raven and/or Cattell performance. Specifically, MFF Time is 
related primarily to Raven performance; MFF Error to both 
Raven and Cattell performance; Trail Making mostly to Cat- 
tell performance; and Visual Search to both Raven and Cat- 
tel 1 performance. 

(3) Whereas MFF Time (for Raven) and MFF Error (for Raven and 
Cattell) appear to related to performance regardless of test 
condition, Trail Making and Visual Search appear to affect 
performance primarily under the verbalization condition 

(C 2 ). 

(4) The Trail Making variable, the only one for which 
significant ethnic group differences were noted, does appear 
to be differentially related to ethnic group performance on 
the Cattell CFT. Specifically, Trail Making is 
significantly correlated with the performance of 

Hispanic subjects under the verbalization test condition, 
but is not significantly correlated with the performance of 
Anglo or Black subjects under this condition. Trail Making 
is related to the performance of Anglo subjects under 
standard administration procedures, but not to the 
performance of Black or Hispanic subjects under this 
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condition. Trail Making time is related to the performance 
of Black subjects under the elaborated feedback condition. 
It is not related to the performance of Anglo subjects under 
C3, and is related significantly to the performance of 
Hispanic subjects only on the classification subtest. 

3.3. Predictive Validity of Raven's SPM and Cattell's CFT 

We are primarily concerned in this section with comparisons of 
the predictive power of the two measures of cognitive ability 
administered under the different test conditions. The criterion 
measures used were Reading and Math standard scores on the 
Comprehensive Test of Basic Skills (CTBS), administered in the Spring 
of the 1982 academic year. Beyond comparing the predictive validity 
of Raven's and Cattell as a function of test condition, we were 
interested in finding out if predictive validity varies as a function 
of ethnicity and/or cognitive style or personality differences. 
Unfortunatley , our restricted sample size, due in part to missing data 
especially on the criterion measures, makes it impossible to perform 
specific comparisons among groups formed on the basis of test 
condition _and ethnicity and cognitive style or personality variables. 
That is, our sample size is insufficient to form groups such as 
"Anglo, Impulsive, C 2 °r "Black, impulsive, C 2 U , and perform 
correlational analyses separately for these groups. 
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Our approach will be as follows: (1) To estimate and compare the 
predictive validity of the Raven and Cattell as a function of test 
condition per se , correlations between the predictor and criterion 
measures will be made for all subjects combined within each test con- 
dition. (2) To estimate the predictive validity of Raven and Cattell 
as a function of test condition, impulsivity and planning. 

Since the analyses performed in the previous section indicated 
that there is little relationship between the Harter Perceived Com- 
petence Scales and either the Raven or Cattell administered under any 
condition, no comparisons of the predictive validity of these measures 
were made based on high and low Harter scale groupings. 

Correlations were computed between Raven and Cattell scores and 
the criterion measures separately for each of the above groupings. 

The above two stage analysis allows us to address the following 
questions: 

(1) Regardless of ethnic group, cognitive style, or personality 
differences, does the predictive validity of the Raven and/or Cattell 
vary as a function of method of test administration? 

(2) Is the predictive validity of Raven and/or Cattell different 
for "impul si ve" children (as measured by MFF Time, MFF Error) under 
different methods of test administration? Is the predictive validity 
of Raven and/or Cattell different for "poor 11 vs. "good" planners (as 
measured by Trail Making and Visual Search) under different methods of 
test administration? 
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3.3.1 Predictive Validity as a Function of Method of Test Admini- 
stration 

Table 15 presents the Pearson product-moment correlation coef- 
ficients and their squared values between the predictor measures 
(Raven, Cattell) and the criterion measures (Reading and Math standard 
scores) separately for each test condition. 

Inspection of Table 15 reveals that the correlation of both pre- 
dictors with both criterion measures is statistically significant for 
all testing conditions. Within each test condition, Raven and Cattell 
predict the criterion measures approximately equally well. Between 
test conditions, however, it can be seen that the predictive power of 
both the Raven and Cattell appears to be slightly under the 
elaborative testing conditions than under standard administsration 
procedures. In terms of percent variance accounted for in the 
criterion measures by the predictor measures, 10% of the variance 
between Raven's Matrices and CTBS Reading is shared under Cj. Under 
C 2 , however, this jumps to 23 percent. Similarly, the shared 
variance between the CFT and Reading is approximately twice greater 
under C3 than under C^. A similar pattern obtains for the 
prediction of Math scores. The shared variance between the Raven and 
Math is approximately three times greater under C 2 than under Cj. 
The CTF predicts Math achievement twice as well under Cg than under 
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Table 15 

Pearson product-Moment Correlations between Predictor Measures 
of Cognitive Ability and Criterion Measrues of School Achievement 



Criterion Measures 



I GSU LrO flQ 1 t 1 Ull 

and Predictor 


Reading 






Math 






r r 


2 (X) 


N 

11 


r 
i 


r 2 (%) 


N 


Cl 














Raven 


.33* 


10.9 


40 


.33* 


inn 

l o.y 




Cattell 


.35* 


12.3 


40 


.35* 


12.3 


40 
















Raven 


.48** 


23.0 


45 


. 54** 


29.2 


44 


Cattell 


.42** 


1 7.6 


44 


.50** 


2 5.0 


44 
















Raven 


.41** 


16.8 


44 


.44** 


19.4- 


45 


Cattell 


.50** 


2 5.0 


43 


.46** 


21 .2 


43 



*p<.05 
**p<.01 
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3.3.2 Predictive Validity as a Function of Method of Test Admini- 
stration, Impulsivity, and Planning Ability 
Within each test condition, subjects were divided into two groups 
(above and below the median score) on the two impulsivity variables 
(MFF Time, MFF Error) and on the three planning variables (Trail Mak- 
ing, and mean and standard deviation of visual search time). Validity 
coefficients were then computed between the precitor and criterion 
measures for each of these groups. The validity coefficients for the 
standard test condition "impulsivity" groups are presented in Table 
16. 

Inspection of this table shows that for the MFF Time, the valid- 
ity coefficients tend to be lower for the "impulsive" subjects, i.e., 
those with below median response times, than for children with re- 
latively longer MFF Times. Both Raven and Cattell predict Math scores 
better for the above median MFF Error group than for the below median 
error gorup. It appears that under standard administration pro- 
cedures (Cj), Raven and Cattell predict the achievement measures 
best for children who take relatively long to respond and for children 
who tend to make a lot of errors on the MFF. The predictive validity 
of Raven and Cattell seems to be poorest for relative fast responders, 
or "impulsive" children as measured by MFF Time. 

Table 17 shows the val idity coefficients for the above and below 
median groupings on the Planning variables for standard administra- 
tion. Inspection of this table indicates that the validity 
coefficients are generally higher for those subjects with relatively 

7j 



74 



Table 16 

Pearson Product-Moment Correlatons between Cognitive Predictor 
Measures and School Achievement Measures for Subject Groupings 
on Impulsivity Variables - Standard Test Condition 



Criterion Measures 



Subject Group 
and Predictor 



Above Med MFF Time 

Raven 

Cattell 
Below Med MFF Time 

Raven 

Cattell 
Above Med MFF Error 

Raven 

Cattell 
Below Med MFF Error 

Raven 

Cattell 



Reading 



Math 



r r 2 (%) N r r 2 (%) 



.35 
.49* 

.31 
.22 

.36 
.32 

.29 
.45* 



9.6 
4.8 

13.0 
10.2 



19 
19 



12.3 21 .38* 
24.0 21 .40* 



.26 
.27 



20 .56** 
20 .43* 



8.4 20 .1 5 
20.3 20 .34 



14.4 21 
16.0 21 



6.8 
7.3 

32.4 
18.5 



19 
19 

20 
20 



2.3 20 
11.6 20 



*p<.05 
**p<.0l 
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Table 17 

Pearson Product-Moment Correlations between Cognitive Predictor 
Measures and Scnool Achievement Measures for Subject Groupings 
on Planning Variables - Standard Test Condition 



Criterion Measures 



Test Condition 
and Predictor 



Readi ng 



Math 



Above Med TM 1 

Raven 

Cattel 1 
Below Med TM 

Raven 

Cattel 1 
Above Med X VST 2 

Raven 

Cattel 1 
Below Med X VST 

Raven 

Cattel 1 
Above Med SD VST 3 

Raven 

Cattel 1 
Below Med SD VST 

Raven 

Cattel 1 



r r 



2 (%) N 



r r 2 {%) 



.29 
.29 

.37 
.42* 

.34 
.32 

.28 
.31 

.33 
.40* 

.30 
.28 



13.7 
17.6 

11.6 
10.2 

7.8 
9.6 

10.9 
16.0 

9.0 
7.8 



20 
20 

20 
20 

20 
20 

19 
19 

21 
21 



8.4 20 .19 
8.4 20 .36 



.48* 
.33 

.13 
.05 

.45* 
.48* 

.17 
.18 

.49* 
.47* 



3.6 20 
13.0 20 



23.0 
10.9 

1.7 

0 

20.2 
23.0 

2.9 
3.2 

24.0 
22.1 



20 
20 

20 
20 

20 
20 

19 
19 

21 
21 



f p<.05 
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1 TM, Tr a i> Making 2 X VST, Mean Visual Search Time 
3 SD VST, Standard Deviation of Visual Search Time 
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fast Trail Making times as opposed to subjects with relative"!:*' slow 
times on this variable. There is little difference in the predictive 
validity of Raven and Cattel for the mean Visual Search groupings with 
respect to Reading. However, both Raven and Cattel 1 seem to predict 
Math achievement better for children with relatively fast visual 
search times as opposed to children with relatively slow visual search 
times* Very similar findings are noted for the above and below median 
groupings on the standard deviation of visual search time. 

To briefly summarize: Under standard administration procedures, 
there appears to be a slight bias in the predictive power of Raven 
and/or Cattel! against fast ("impulsive") responders, as measured by 
MFF Time, and against "poor planners," as measured by all three 
planning variables. 

Table 18 shows the validity coefficients for the MFF groupings of 
subjects who were administered Raven and Cattel! under the 
verbalization testing condition (C2). 

Inspection of Table 18 reveals that the predictive validity of 
Raven and Cattel! is much higher for "impulsive" children as measured 
by MFF Time than it is for this group under standard administration 
procedures (Cj). The validity coefficients are also slightly high- 
er, on the average, for the above median MFF Time group than they are 
for this group under Cj. A similar pattern of coefficients will be 
noted for the above median MFF Error group here as was noted for this 
group under Cj« That is, both Raven and Cattel! tend to 
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Table 18 

Pearson Product-Moment Correlations between Cognitive Predictor 
Measures and School Achievement Measures for Subject Groupings 
on Impulsivity Variables - Verbalizati on Test Condition 



Subject Group 
and Predictor 



Criterion Measures 



Reading 



M?th 



Above Med MFF Time 

Raven 

Cattell 
Below Med MFF Time 

Raven 

Cattell 
Ab ove Med MFF Error 

Raven 

Cattell 
Below Med MFF Error 

Raven 

Cattell 



.57** 
.44* 

.42* 
.37* 

.22 
.20 

.67** 
.42* 



~2 (%) 

32.5 
19.4 

17.6 
13.7 

4.8 
4.0 

44.9 
17.6 



2 (%) 



21 
20 

24 
24 

20 
20 

25 
24 



.59** 
.57** 

.52** 
.46* 

.51** 
.45* 

.48** 
.41* 



34.8 
32.5 

27.0 
21 .2 

26.0 
, 20.3 

2-i.O 
16.8 



20 
20 

24 
24 

20 
20 

24 
24 



*p<.05 
**p<.0l 
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predict Math achievement well for this group, but not reading achieve- 
ment. However, for the below median MFF error group, the validity 
coefficients are much higher for Math achievement and (for Raven) 
reading achievement Vun they are for this group under C^, 

Table 19 presents the validity coefficients for the Planning 
variable groupings of subjects admini stared Raven and Cattel 1 under 
C 2 . Compared to these groupings under C-j , the validity 
coefficients are, on the averaye, much higher for all groups. 
Especially noteworthy are the higher coefficients reported for the 
above median Trail Making group (particularly for math achievement), 
and for the above median mean and SD Visual Search groups (again, 
particularly for math achievement). 

Comparisons of the predictive validity of Raven and Cattell 
administered under standard and verbalization testing conditions as a 
function of "impulsi vity" and "planning" leads to the following 
general conclusions: (1) Predictive validity is higher under 
verbalization than under standard testing procedures, and (2) 
verbalization appears to reduce or eliminate the predictive bias de- 
monstrated under standard testing procedures against "impulsive" 
children (as measured by MFF Time) and "poor planners," as measured by 
the Trail Making and Visual Search Tasks. The improvement in pre- 
diction is especially great for math achievement. 

Table 2Q presents the validity coefficients for the impulsivity 
variable groupings for subjects in the elaborated feedback condition 

(C 3 ). 
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Table 19 

Pearson Product-Moment Correlations between Cognitive Predictor 
Measures and School Achievement Measures for Subject Groupings 
on Planning Variables - Verbalization Test Condition 
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Subject Group 
and Predictor 



Above Med TM 



9 
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Criterion Measures 



Reading Math 



r r 2 (%) N r r 2 (%) 



Raven .3 7* 13. 7 21 .60** 36.0 21 

Cattell .22 4.8 21 .40* 16.0 21 

Below Med TM 

Raven .57** 32. 5 24 .53** 28.1 23 

Cattell .53** 28.1 23 .54** 29.2 23 
Above Med I VST 

Raven .33 10.9 23 .47* 22.1 22 

Cattell .28 7.8 22 .42* 1 7.6 22 

Below Med ~X VST 

Raven .61** 37.2 22 .65** 42.3 22 

Cattell .52 27.0 22 .56 31 .4 22 

Above Med SD VST 

Raven .48* 23.0 23 .63** 39.7 22 

Cattell .30 9.0 22 .45* 20.3 22 

Below Med SD VST 

Raven .46* 21 .2 22 .42* 17.6 22 

Cattell .56** 31.4 22 .54 29.2 22 

*p<.05 
**p< . 01 
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Table 20 

Pearson Product-Moment Correlations between Cognitive Predictor 

Measures and School Achievement Measures for Subject Groupings 

on Impulsivity Variables - Elaborated Feedback Te st Condition 



Subject Group 
and Predictor 



Above Med MFF Time 

Raven 

Cattell 
Below Med MFF Time 

Raven 

Cattel 1 
Above Median MFF Error 

Raven 

Cattell 
Below Median MFF Error 

Raven 

Cattell 



-p<.05 
**p<.01 



Criterion Measures 



Reading 



Math 



r 2 (%) 



r 2 (%) 



.37 



.35* 
.35* 

.40* 
.36 

.31 
.51** 



13.7 20 .61 



** 



12.3 
12.3 

16.0 
13.0 

9.6 
25.0 



25 
25 

21 
21 

24 
24 



.09 
.20 

.07 
.22 

.57** 
.54** 



37.2 20 



,54** 29.2 20 .53** 28.1 20 



1.0 
4.0 

0.5 
4.8 

32.5 
29.2 



23 
23 

20 
20 

23 
23 
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Inspection of the table reveals that, as for the subjects in C lf the 
validity coefficients for relatively fast responders as measured by 
MFF Time ("impul si ves M ) are much lower than for relatively slow 
responders. This is especially true with respect to the prediction of 
math achievement. Prediction of reading achievement is slightly 
greater for the relatively slow responding group as well. Unlike the 
verbalization test condition, elaborated feedback does not seem to 
compensate for "impul si veness" as measured by MFF time. 

Prediction is greater for the below median MFF error group than 
for the above medican MFF error group. Again, this obtains 
particularly for math achievement. Whereas Raven and Cattell under 
Ci Predict math achievement better for subjects who make relatively 
many errors on the MFF, under C 3 they predict math achievement best 
for subjects who tend to make relatively few errors on the MFF. As 
opposed to the verbalization condition, in which math achievement was 
pr^^ 'ed approximately equally as well for both high and low error 

, the standard and elaborated feedback conditions over- or 
under-predicts for one or the other of these groups. 

Table 21 presents the validity coefficients for the planning 
variable groupings for subjects in the elaborated feedback condition. 

The coefficients presented in this table are similar to those 
presented in Table 19 for the verbalization condition subjects. Pre- 
diction is greater for subjects hwo have relatively long Trail Making 
times ("poor planners") than it is for this group under standard tes- 
ting procedures, and is higher for the below median 
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Table 21 

Pearson Product-Moment Correlations between Cognitive Predictor 
Measures and School Achievement Measures for Subject Groupings 
on Planning Variables - Elaborated Feedback Test Condition 



Subject Group 
and Predictor 



Criterion Measures 



Reading Math 



r 2 (%) n r r 2 (%) 



Above Med TM 

Raven .33 10.9 24 .43* 18.5 23 

Cattell .44* 19.4 24 .36* 13.0 23 
Below Med TM 

Raven .50** 25.0 21 .46* 21 .2 20 

Cattell .54** 29.2 21 .55** 30.3 20 
Above Med X VST 

Raven .32 10.2 22 .48* 23.0 20 

Cattell .58** 33.6 22 .54** 29.2 20 
Below Med X VST 

Raven .60** 36.0 23 .48* 23.0 23 

Cattell .55* 30.3 23 .49** 24.0 23 
Above Med SD VST 

Raven .30 9.0 22 .57** 32.5 21 

Cattell .58** 33 . 6 2 2 . 63** 40. 0 21 
Below Med SD VST 

Raven .57** 32.5 23 .40* 16.0 22 

Cattell .56** 31.4 23 .50 25.0 22 



*P<.05 

**p<.01 So 
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trail making time group ("good planners") compared to this group under 
Cj as wel 1 . 

As for the above median Visual Search time group in the 
verbalization condition, prediction of math achievement is con- 
siderably higher than under Cj. Similarly, prediction of both 
reading and math achievement is higher, on the average, for the below 
median visual search time and both SD Visual Search time groupings 
than it is for these groups under standard administration procedures. 

Many of the same conclusions can be drawn concerning the pre- 
dictive validity of Raven and Cattell as a function of planning abil- 
ity when these measures are administered under elaborated feedback 
conditions as when they are administered under the verbalization 
condition, 

3,3.3 Summary 

The results of the predictive validity analyses presented here 
indicate that elaborative testing procedures tend to increase the 
power of measures of cognitive ability to predict school achievement. 
In addition, both elaborative conditions appear to compensate for the 
"bias" in prediction noted under standard administration procedures 
for specific groups of subjects, i.e., for "poor planners" as measured 
by the Trail Making and Visual Search Tasks. Only the verbalization 
test condition appears to compensate for "impulsiveness" as measured 
by MFF Time, however. 
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Chapter IV 
Summary 

4.0 Organization 

The rather extensive results presented in the previous chapter 

will not be reiterated in this section. Rather, the approach will be 

to highlight and discuss the findings as they relate to the specific 

questions which guided the research. 

4c 1 Research Question 1 

The initial research question and its corollary was: 

Does the dynamic assessment approach yield information which 
is a more appropriate indicator of cognitive competence than 
that obtained by traditional, standard approaches based on 
static test theory? 

Corollary: Do specific testing conditions differentially 
affect the performance of children of different social an- 
d/or ethnic backgrounds? 
The results suggest a positive response to the initial question 
and a negative response to its corollary. The initial expectation was 
that differences in performance on both the Raven and Cattell tests 
would be detected between the three ethnic groups under the standard 
procedure of administering these tests. A further expectation was 
that under dynamic testing procedures all groups would improve, with 
lower Black and Hispanic performance improving to the extent that the 
differences found under standard testing procedures would be reduced. 
That is, the dynamic assessment apporoach would have a compensating 
effect. These expectations were only partially fulfilled. 

90 
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The testing condition which seemed to be most effective in 
increasing performance of all groups of children was the condition 
involving verbalization. Under this condition significant 
improvements on both the Raven Matrices and the Cattel 1 Culture Fair 
Test were noted when compared with performance under the standard 
testing procedures. No testing condition by ethnic classification 
interaction was detected for either the Raven or Cattel 1. Thus, 
verbalization was effective in improving performance regardless of 
ethnic classification. On the other hand, if one compares performance 
of the Black group with that of the Anglo group on the Raven as well 
as on the Cattell, it will be noted that Black performance under 
verbalization is equivalent to Anglo performance under standard 
testing procedures. The same obtains for the Hispanic group for the 
Cattell but not for the Raven. One could conclude that a type of 
compensatory effect for initially observed differences between these 
groups was brought about by subject verbalization. Using different 
modes or test administration for some groups as opposed to others in 
order to bring about this compensation may not be a justified 

practice, however. 

Analysis of the effects of the testing conditions on subgroups of 
Raven items indicates that performance on the relatively difficult 
reasoning items was most affected by verbalization. These items 
require analogical reasoning and cannot be solved by perceptual 
processes. Performance of all ethnic groups increased on these items 
and no ethnic classification x testing condition interaction was 
found. Nonetheless, the type of compensatory effect noted for the 
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total scores mentioned above was found. Under verbalization, the 
performance of the Black and Hispanic children reached the performance 
level found for Anglo children under standard testing conditions. 

Analyses involving the Cattel 1 show that verbalization was ef- 
fective for only certain subgroupings of the test: matrices and con- 
ditions. For the former, verbalization performance was higher than 
performance under standard conditions whereas for the latter it was 
higher than either standard or elaborated feedback conditions. These 
results are consistent with those obtained with the Raven matrices as 
the dependent variable. 

The question of why verbalization effectively improves per- 
formance over that obtained in either standard testing or in the con- 
dition involving elaborated feedback is important. It should be re- 
membered that in the verbalization condition, the subject was not 
given any feedback concerning the correctness of his/her response. 
The only difference between verbalization and the procedure followed 
under the standard testing condition was that the subject was asked to 
overtly describe the task at hand and his/her thinking processes as 
the task was solved. Overt relevant verbalization apparently brings 
about modifications in the central processing of information. 

This can be depicted as follows: 
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Stimulus Input 



Elaboration 




Involves Modifications in 
Central Processing 



Modifications in central processing could be brought about by 
several factors operating independently or conjointly. These include 
planfulness, exactness, self-regulation, flexibility, and sharpening 
of responses. In addition, anxiety could be reduced and position ori- 
entation to the test situation increased. Abundant evidence for this 
has been cited elsewhere (see Carlson & Wiedl , 1980; Carlson, Bethge, 
& Wiedl, 1981; Dillon, 1981). 
4.2 Research Question 2 

The second research question was: 

Why and for whom is the dynamic assessment approach ef- 
fective? Or, what are the sources of intraindi vidua! varia- 
tion, and how may they be accounted for by personality 
and cognitive style variables? 
The results did not yield clear answers to these questions. Pre- 
vious research led us to the expectation that the efficacious dynamic 
assessment approach(es) would be compensatory for individuals whose 
performance was poor on "nontarget" variables such as impulsivity, 
planning or perceived competence. More specifically, it was expected 



88 

that gains in performance could be at least partially attributable to 
the reduced negative effect of the nontarget variable in the ef- 
ficacious testing condition(s). In addition, it was expected that 
differences between the ethnic groups in the selected per- 
sonality/cognitive style variables would be demonstrated and help ex- 
plain differences in cognitive performance as assessed under standard 
testing conditions. 

General comparisons between ethnic groups on the Matching 
Familiar Figures Task indicated that Anglos tended (the difference 
only approached statistical significance) to be more accurate than 
either the Black or Hispanic children. No significant differences in 
time taken to solution were found. In planning, Anglos tended to be 
slightly "better planners" than did Blacks or Hispanics. No signific- 
ant differences or trends towards differences were found on any of the 
perceived competence scales, allowing the conclusion that re- 
gardless of ethnicity the children's perceptions of their cognitive, 
social and physical skills were the same. 

The most consistent correlations with performance on either the 
Cattell or Raven measures was for errors made on the Matching Familiar 
Figures Test. Here, however, the relationships tended to be con- 
sistent regardless of testing condition. Hence no interaction between 
this varible and performance under differing testing conditions was 
detected. Apparently, at least for the subjects involved in this 
study, those modifications in central processing which were apparently 
brought about by the verbalization condition are not systematically 
related to any of the cognitive style/personalities variables 

9 

assessed. ' 
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4.3 Research Question 3 

The third research question guiding the study was: 

For what reason should dynamic assessment procedures be em- 
pi oyed? 

This question is broad. It can be related to specific issues of 
predictive validity, indications of how cognitive and perceptual 
abilities should be assessed, or indications of approaches which might 
be useful in certain instructional settings. Although the details of 
the results concerning predictive validity are given in sections 
3.3.1, 3.3.2, and 3.3.3 of the previous chapter, further comment is 
appropriate. 

The systematic effect of subject verbalization of increasing per- 
formance on both the Raven and Cattdl implies that modi f icatirns in 
central processing of information were brought about. Levels of per- 
formance were reached which more closely approximate cognitive com- 
petence than estimated by either standard assessment or the approach 
which involved extensive feedback. Generally speaking, the predictive 
validities of the Raven and Cattell are moderate. Confirmation of 
this was gained through the correlations of these measures with the 
mathematics and reading achievement scores. Correlations between both 
the Raven and Cattell and the achievement measurss increased when the 
former tests were administered under the verbalization condition. Ap- 
parently the gains in predictive validity involve certain aspects of 
individual differences in "impul si vity" and planning. Neither the 
Raven nor Cattell have better than marginal predictive validity for 
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"impulsive" responders or poor planners when the tests are 
administered in the traditional manner. When verbalization is em- 
ployed, however, the predictive validities for these individuals is 
increased. Thus, the gains in overall predictive validity noted under 
verbalization are due largely to the improved prediction gained for 
those persons classified as "impulsive" and poor planners; 
characteristics, by the way, for which differences between the ethnic 

groups was found. 

These results suggest that overt verbalization leads not only to 
more accurate assessment of cognitive and perceptual functioning but 
that the approach also yields better prediction of school achievement. 
What would appear to be a bias toward under prediction for poor plan- 
ners and 'Mmpulsives" (again, not unrelated to the ethnic clas- 
sification Of the individual) is compensated for by a testing approach 
such as that used in verbalization. A further impl ication 'of the 
results is that employing active, overt verbalization in the cognitive 
school tasks may lead to increased achievements and compensate for 
certain non-cog-niti ve individual difference factors. 
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